15 research outputs found
Recover Subjective Quality Scores from Noisy Measurements
Simple quality metrics such as PSNR are known to not correlate well with
subjective quality when tested across a wide spectrum of video content or
quality regime. Recently, efforts have been made in designing objective quality
metrics trained on subjective data (e.g. VMAF), demonstrating better
correlation with video quality perceived by human. Clearly, the accuracy of
such a metric heavily depends on the quality of the subjective data that it is
trained on. In this paper, we propose a new approach to recover subjective
quality scores from noisy raw measurements, using maximum likelihood
estimation, by jointly estimating the subjective quality of impaired videos,
the bias and consistency of test subjects, and the ambiguity of video contents
all together. We also derive closed-from expression for the confidence interval
of each estimate. Compared to previous methods which partially exploit the
subjective information, our approach is able to exploit the information in
full, yielding tighter confidence interval and better handling of outliers
without the need for z-scoring or subject rejection. It also handles missing
data more gracefully. Finally, as side information, it provides interesting
insights on the test subjects and video contents.Comment: 16 pages; abridged version appeared in Data Compression Conference
(DCC) 201
SpatioTemporal Feature Integration and Model Fusion for Full Reference Video Quality Assessment
Perceptual video quality assessment models are either frame-based or
video-based, i.e., they apply spatiotemporal filtering or motion estimation to
capture temporal video distortions. Despite their good performance on video
quality databases, video-based approaches are time-consuming and harder to
efficiently deploy. To balance between high performance and computational
efficiency, Netflix developed the Video Multi-method Assessment Fusion (VMAF)
framework, which integrates multiple quality-aware features to predict video
quality. Nevertheless, this fusion framework does not fully exploit temporal
video quality measurements which are relevant to temporal video distortions. To
this end, we propose two improvements to the VMAF framework: SpatioTemporal
VMAF and Ensemble VMAF. Both algorithms exploit efficient temporal video
features which are fed into a single or multiple regression models. To train
our models, we designed a large subjective database and evaluated the proposed
models against state-of-the-art approaches. The compared algorithms will be
made available as part of the open source package in
https://github.com/Netflix/vmaf